Variable reliability and capacity data center design

ABSTRACT

A design for the electrical infrastructure of a data center enables two different configurations to be utilized, including a distributed, redundant configuration that provides higher reliability and a non-redundant configuration that provides higher capacity. In the distributed, redundant configuration, each server in the data center draws power from at least two different power supply systems. This enables load shifting when a power supply system becomes unavailable, which can have the effect of increasing server reliability. In the non-redundant configuration, each server in the data center draws power from only one power supply system. Load shifting is not utilized in the non-redundant configuration, which eliminates the need to maintain reserve capacity and thereby increases capacity. Advantageously, it is possible to switch between these two configurations without making any internal changes to the data center other than modifying connections between sets of server racks and power buses.

CROSS-REFERENCE TO RELATED APPLICATIONS

N/A

BACKGROUND

A data center is a physical facility that is used to house computersystems and associated components. A data center typically includes alarge number of servers, which may be stacked in racks that are placedin rows. A colocation center (which is sometimes referred to simply as a“colo”) is a type of data center where equipment, space, and bandwidthare available for rental to customers.

The electrical infrastructure of a data center (such as a colocationcenter) includes a connection to the main power grid, which is typicallyprovided by the local utility company. The electricity from the localutility company is typically delivered with a medium voltage. Themedium-voltage electricity is then transformed by one or moretransformers to low voltage for use within the data center. To ensureuninterrupted operation even in the case of a large-scale power outage,data centers are typically connected to at least one backup generator.Electricity from the backup generator may be delivered at low voltage,or it may be delivered at medium voltage and then transformed to lowvoltage for use within the data center. The low-voltage electricity isdistributed to endpoints through one or more Uninterrupted Power Supply(UPS) systems and one or more power distribution units (PDUs). A UPSsystem provides short-term power when the input power source fails andprotects critical components against voltage spikes, harmonicdistortion, and other common power problems. A PDU includes multipleoutputs that are designed to distribute electric power to racks ofcomputers and networking equipment located within a data center.

The electrical infrastructure of a data center can utilize a pluralityof different cells. Each of the cells can include its own power supplysystem. In this context, the term “power supply system” may refer to oneor more components that provide a source of power to at least some ofthe servers and/or other components in the data center. A power supplysystem may include one or more of the components described previously(e.g., a connection to the main grid, a backup generator, one or moretransformers, a UPS system, and one or more PDUs).

SUMMARY

In accordance with one aspect of the present disclosure, a method isdisclosed that includes providing a plurality of power supply systems, aplurality of sets of server racks, and a plurality of power buses. Theplurality of power supply systems are configured to supply electricalpower to at least a section of a data center. The plurality of powerbuses are configured to conduct the electrical power from the pluralityof power supply systems to the plurality of sets of server racks. Themethod also includes connecting the plurality of sets of server racksand the plurality of power buses in accordance with a first connectionpattern to place the section of the data center in a distributed,redundant configuration. Each server rack in the plurality of sets ofserver racks is connected to two different power supply systems when theplurality of sets of server racks and the plurality of power buses areconnected in accordance with the first connection pattern. The methodalso includes connecting the plurality of sets of server racks and theplurality of power buses in accordance with a second connection patternto place the section of the data center in a non-redundantconfiguration. Each server rack in the plurality of sets of server racksis connected to only one power supply system when the plurality of setsof server racks and the plurality of power buses are connected inaccordance with the second connection pattern.

The distributed, redundant configuration may provide higher serverreliability than the non-redundant configuration. The non-redundantconfiguration may provide higher power capacity than the distributed,redundant configuration.

The method may further include transitioning between the distributed,redundant configuration and the non-redundant configuration withoutmaking any internal changes to the data center other than modifyingconnections between the plurality of sets of server racks and theplurality of power buses.

Connecting the plurality of sets of server racks and the plurality ofpower buses in accordance with the first connection pattern may includeconnecting each server rack in the plurality of sets of server racks totwo different power buses that are connected to two different powersupply systems. Connecting the plurality of sets of server racks and theplurality of power buses in accordance with the second connectionpattern may include connecting each server rack in the plurality of setsof server racks to only one power bus.

The plurality of power supply systems may operate at full capacitywithout maintaining any reserve capacity when the plurality of sets ofserver racks and the plurality of power buses are arranged in thenon-redundant configuration. The plurality of power supply systems maymaintain reserve capacity during normal operation when the plurality ofsets of server racks and the plurality of power buses are arranged inthe distributed, redundant configuration. The distributed, redundantconfiguration may enable load shifting when a power supply systembecomes unavailable.

The method may further include connecting equal numbers of power busesto different power supply systems of the plurality of power supplysystems.

In accordance with another aspect of the present disclosure, a system isdisclosed that includes a plurality of power supply systems that areconfigured to supply electrical power to at least a section of a datacenter. The system also includes a first plurality of sets of serverracks and a first plurality of power buses that are configured toconduct the electrical power from at least some of the plurality ofpower supply systems to the first plurality of sets of server racks. Thefirst plurality of sets of server racks and the first plurality of powerbuses are connected in accordance with a first connection pattern tocreate a distributed, redundant configuration. The system also includesa second plurality of sets of server racks and a second plurality ofpower buses that are configured to conduct the electrical power from atleast some of the plurality of power supply systems to the secondplurality of sets of server racks. The second plurality of sets ofserver racks and the second plurality of power buses are connected inaccordance with a second connection pattern to create a non-redundantconfiguration.

The distributed, redundant configuration may provide higher serverreliability than the non-redundant configuration. The non-redundantconfiguration may provide higher power capacity than the distributed,redundant configuration.

Each server rack in the first plurality of sets of server racks may beconnected to two different power supply systems. Each server rack in thesecond plurality of sets of server racks may be connected to only onepower supply system.

The first plurality of power supply systems may maintain reservecapacity during normal operation. The second plurality of power supplysystems may operate at full capacity without maintaining any reservecapacity.

In accordance with another aspect of the present disclosure, a system isdisclosed that includes a plurality of power supply systems that areconfigured to supply electrical power to at least a section of a datacenter. The system also includes a plurality of sets of server racks anda plurality of power buses that are configured to conduct the electricalpower from the plurality of power supply systems to the plurality ofsets of server racks. Each server rack in the plurality of sets ofserver racks has two connections to a power bus. Each server rack in theplurality of sets of server racks is connected to only one power bus.Adjacent server racks in a row of server racks are connected todifferent power buses.

The plurality of power supply systems may operate at full capacitywithout maintaining any reserve capacity. Equal numbers of power busesmay be connected to different power supply systems of the plurality ofpower supply systems.

In some embodiments, a first power bus runs parallel to a first side ofthe row of server racks, and a second power bus runs parallel to asecond side of the row of server racks. Every other server rack in therow of server racks may be connected to a different one of the firstpower bus and the second power bus.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Additional features and advantages will be set forth in the descriptionthat follows. Features and advantages of the disclosure may be realizedand obtained by means of the systems and methods that are particularlypointed out in the appended claims. Features of the present disclosurewill become more fully apparent from the following description andappended claims, or may be learned by the practice of the disclosedsubject matter as set forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otherfeatures of the disclosure can be obtained, a more particulardescription will be rendered by reference to specific embodimentsthereof which are illustrated in the appended drawings. For betterunderstanding, the like elements have been designated by like referencenumbers throughout the various accompanying figures. Understanding thatthe drawings depict some example embodiments, the embodiments will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates an example showing how servers and power buses can bearranged in a section of a data center in order to enable differentconfigurations to be utilized.

FIG. 1A illustrates an example showing how the server racks within a setof server racks can be connected to power buses in order to produce adistributed, redundant configuration.

FIG. 1B illustrates an example showing how the server racks within a setof server racks can be connected to power buses in order to produce anon-redundant configuration.

FIG. 2 illustrates an example of a method that can be performed in orderto facilitate a distributed, redundant configuration and a non-redundantconfiguration for the electrical infrastructure that is utilized in asection of a data center.

FIG. 3 illustrates examples of components that can be included in thepower supply systems of various cells in order to facilitate adistributed, redundant configuration.

FIG. 4 illustrates an example of a data center in which differentsections utilize different configurations.

DETAILED DESCRIPTION

The present disclosure is generally related to a design for theelectrical infrastructure of a data center that enables two differentconfigurations to be utilized without requiring any significant changesto the underlying infrastructure components. These configurations caninclude a first configuration that provides higher reliability at thecost of lower capacity, and a second configuration that provides highercapacity at the cost of lower reliability. The first configuration maybe referred to herein as a “higher reliability, lower capacity”configuration, and the second configuration may be referred to herein asa “higher capacity, lower reliability” configuration.

In this context, the term “reliability” can refer to the ability of theelectrical infrastructure of a data center to supply electrical power toservers and other electrical components in a data center in a consistentand dependable way, such that the servers and other electricalcomponents are able to remain in operating condition for a highpercentage of the time. In other words, reliability can be considered tobe synonymous with availability. On the other hand, the term “capacity”can refer to the amount of electrical power that can be supplied by theelectrical infrastructure of a data center. A higher capacity, lowerreliability configuration may be capable of supplying a greater amountof electrical power than a higher reliability, lower capacityconfiguration. However, a higher reliability, lower capacityconfiguration may be capable of supplying electrical power moreconsistently and dependably than a higher capacity, lower reliabilityconfiguration. More specifically, a higher reliability, lower capacityconfiguration may be capable of producing higher server availabilitythan a higher capacity, lower reliability configuration.

As noted previously, the electrical infrastructure of a data center canutilize a plurality of different cells, and each of the cells caninclude its own power supply system. For the higher reliability, lowercapacity configuration, the electrical infrastructure can utilize adistributed, redundant architecture in which each server in the datacenter draws power from at least two different power supply systems. Ifa power supply system becomes unavailable, the load that was beingprovided by the now unavailable power supply system can be shifted toone or more other power supply systems. In this way, the servers thatwere being supplied by the now unavailable power supply system cancontinue to receive power. Therefore, the use of a distributed,redundant architecture enables load shifting, which can have the effectof increasing the reliability of servers in a data center.

Load shifting in the manner just described involves increasing theamount of power that is supplied by at least some of the other powersupply systems in the data center. To reduce the likelihood that loadshifting would cause any of the components of the other power supplysystems to become overloaded, the electrical infrastructure can bedesigned so that it operates with a certain amount of reserve capacityunder normal circumstances. In other words, under normal circumstances(when all power supply systems are operational), the utilization of adata center can be limited so that the servers' total power consumptiondoes not exceed a threshold level that is lower than the electricalinfrastructure's total capacity. The difference between this thresholdlevel and the total capacity can be thought of as the reserve capacity.When a power supply system fails and the load that was being supplied bythe now unavailable power supply system is shifted to other power supplysystems, these other power supply systems can tap into their reservecapacity to handle the additional load.

As noted above, in addition to the higher reliability, lower capacityconfiguration that was just described, another possible configurationfor the electrical infrastructure of a data center is a higher capacity,lower reliability configuration. The higher capacity, lower reliabilityconfiguration does not utilize load shifting. Therefore, the highercapacity, lower reliability configuration can utilize a non-redundantarchitecture in which each server in the data center draws power fromonly one power supply system. Because load shifting is not utilized,there is no need to maintain any reserve capacity. Eliminating the needto maintain reserve capacity effectively increases the capacity of thedata center's electrical infrastructure. In other words, the utilizationof the data center can be higher than it would be if reserve capacityhad to be maintained. However, this increased capacity is achieved atthe expense of lower reliability. Without the possibility of loadshifting, the failure of a power supply system causes the servers thatwere being supplied by the now unavailable power supply system to losepower and experience an outage.

One aspect of the present disclosure includes techniques for arrangingthe components of a data center's electrical infrastructure so that itis possible to easily switch between the higher reliability, lowercapacity configuration that utilizes a distributed, redundantarchitecture and the higher capacity, lower reliability configurationthat utilizes a non-redundant architecture. Advantageously, thetechniques disclosed herein make it possible to switch between these twoconfigurations without making any significant changes to the underlyinginfrastructure components.

FIG. 1 illustrates an example showing how servers and power buses can bearranged in a section of a data center 100 in order to facilitate thetechniques disclosed herein. In this example, it will be assumed thatthere are four cells that are configured to supply power to servers in aparticular section of the data center 100. These cells will be referredto as cell A, cell B, cell C, and cell D. Each of the cells includes apower supply system. In particular, cell A includes a power supplysystem 102 a, cell B includes a power supply system 102 b, cell Cincludes a power supply system 102 c, and cell D includes a power supplysystem 102 d.

The data center 100 shown in FIG. 1 includes a plurality of sets ofserver racks 106 a-x, 108 a-x. In this context, the term “rack” canrefer to a physical structure that holds servers. A set of server rackscan include a plurality of server racks.

The data center 100 shown in FIG. 1 includes a plurality of power buses.More specifically, the data center 100 includes a first set of powerbuses 110 a-x that are connected to the power supply system 102 a ofcell A, a second set of power buses 112 a-x that are connected to thepower supply system 102 b of cell B, a third set of power buses 114 a-xthat are connected to the power supply system 102 c of cell C, and afourth set of power buses 116 a-x that are connected to the power supplysystem 102 d of cell D. In this context, the term “power bus” can referto an electrical conductor that is configured to conduct electricalpower. A power bus can alternatively be referred to as a busbar. Thepower buses 110 a-x, 112 a-x, 114 a-x, 116 a-x shown in FIG. 1 areconfigured to conduct electrical power from the power supply systems 102a-d of cells A, B, C, and D to the sets of server racks 106 a-x, 108a-x.

The sets of server racks 106 a-x, 108 a-x and the sets of power buses110 a-x, 112 a-x, 114 a-x, 116 a-x are arranged so that each set ofserver racks is accessible to two power buses that are connected todifferent power supply systems. In some embodiments, a set of serverracks is accessible to two power buses if the set of server racks can beconnected to both of the power buses without having to physically move(or reposition) the set of server racks. In other embodiments, a set ofserver racks is accessible to two power buses if the set of server rackscan be connected to both of the power buses without having to physicallymove (or reposition) the set of server racks or either of the powerbuses.

In the depicted example, each set of server racks is located between twopower buses that are connected to different power supply systems. Forexample, the first set of server racks 106 a is located between a powerbus 110 a that is connected to the power supply system 102 a of cell Aand a power bus 112 a that is connected to the power supply system 102 bof cell B. This makes it possible for each server rack in the set ofserver racks 106 a to be connected to one or both of the power buses 110a, 112 a, as will be discussed in greater detail below. Connecting aserver rack to a power bus can be accomplished via one or moreelectrical cables.

There are equal numbers of power buses that are connected to thedifferent power supply systems 102 a-d of the various cells. Morespecifically, in the depicted example there are four cells. Thus, of thetotal number of power buses, one fourth are power buses 110 a-x that areconnected to the power supply system 102 a of cell A, one fourth arepower buses 112 a-x that are connected to the power supply system 102 bof cell B, one fourth are power buses 114 a-x that are connected to thepower supply system 102 c of cell C, and one fourth are power buses 116a-x that are connected to the power supply system 102 d of cell D.

Of course, the particular arrangement of power supply systems 102 a-d,sets of server racks 106 a-x, 108 a-x, and sets of power buses 110 a-x,112 a-x, 114 a-x, 116 a-x shown in FIG. 1 is just an example and shouldnot be interpreted as limiting the scope of the present disclosure.There are other alternative arrangements for the power supply systems102 a-d, the sets of server racks 106 a-x, 108 a-x, and the sets ofpower buses 110 a-x, 112 a-x, 114 a-x, 116 a-x that are consistent withthe present disclosure. In other words, the power supply systems 102a-d, the sets of server racks 106 a-x, 108 a-x, and the sets of powerbuses 110 a-x, 112 a-x, 114 a-x, 116 a-x can be connected in differentways and still utilize the techniques disclosed herein.

The arrangement of the sets of server racks 106 a-x, 108 a-x and thepower buses 110 a-x, 112 a-x, 114 a-x, 116 a-x shown in FIG. 1facilitates either a higher reliability, lower capacity configurationthat utilizes a distributed, redundant architecture or a highercapacity, lower reliability configuration that utilizes a non-redundantarchitecture. More specifically, if the sets of server racks 106 a-x,108 a-x and the power buses 110 a-x, 112 a-x, 114 a-x, 116 a-x areconnected in accordance with a first connection pattern, this produces ahigher reliability, lower capacity configuration that utilizes adistributed, redundant architecture. Alternatively, if the sets ofserver racks 106 a-x, 108 a-x and the power buses 110 a-x, 112 a-x, 114a-x, 116 a-x are connected in accordance with a second connectionpattern, this produces a higher capacity, lower reliabilityconfiguration that utilizes a non-redundant architecture. Eitherconfiguration can be achieved without any internal changes to the datacenter 100 other than changing the connections between the sets ofserver racks 106 a-x, 108 a-x and the power buses 110 a-x, 112 a-x, 114a-x, 116 a-x. For example, either configuration can be achieved withoutchanging or moving any of the sets of server racks 106 a-x, 108 a-x orthe power buses 110 a-x, 112 a-x, 114 a-x, 116 a-x. Some externalchanges may be needed as well, such as increasing the size of theutility feed to the data center 100 in order to produce the highercapacity, lower reliability configuration.

For simplicity, in the discussion that follows the higher reliability,lower capacity configuration that utilizes a distributed, redundantarchitecture may be referred to simply as a “distributed, redundantconfiguration.” Similarly, the higher capacity, lower reliabilityconfiguration that utilizes a non-redundant architecture may be referredto simply as a “non-redundant configuration.”

FIG. 1A illustrates an example showing how the first set of server racks106 a can be connected to the power buses 110 a, 112 a in order toproduce the distributed, redundant configuration. As shown, the firstset of server racks 106 a includes a plurality of server racks 106a(1)-106 a(20). The server racks 106 a(1)-106 a(20) are arranged in arow. The power bus 110 a runs parallel to a first side of the row ofserver racks 106 a(1)-106 a(20). The power bus 112 a runs parallel to asecond side of the row of server racks 106 a(1)-106 a(20).

The server racks 106 a(1)-106 a(20) are arranged in accordance with aconnection pattern. In this context, the term “connection pattern” canrefer to an arrangement of sequence of connections between the serverracks 106 a(1)-106 a(20) and the power buses 110 a, 112 a. In thedepicted example, each of the server racks 106 a(1)-106 a(20) isconnected to both the power bus 110 a that is connected to the powersupply system 102 a of cell A and the power bus 112 a that is connectedto the power supply system 102 b of cell B.

For example, referring to the server rack 106 a(1) that is at the top ofthe first set of server racks 106 a, there is a connection 118 a betweenthat server rack 106 a(1) and the power bus 110 a that is connected tothe power supply system 102 a of cell A. In addition, there is also aconnection 120 a between that server rack 106 a(1) and the power bus 112a that is connected to the power supply system 102 b of cell B.Similarly, referring to the next server rack 106 a(2), there is aconnection 118 b between that server rack 106 a(2) and the power bus 110a that is connected to the power supply system 102 a of cell A. Inaddition, there is also a connection 120 b between that server rack 106a(2) and the power bus 112 a that is connected to the power supplysystem 102 b of cell B. There are similar connections 118 c-t betweenthe server racks 106 a(3)-(20) and the power bus 110 a that is connectedto the power supply system 102 a of cell A, and similar connections 120c-t between the server racks 106 a(3)-(20) and the power bus 112 a thatis connected to the power supply system 102 b of cell B.

The connections 118 a-t, 120 a-t shown in FIG. 1A between the first setof server racks 106 a and the power buses 110 a, 112 a arerepresentative of the connections that can be made between the varioussets of server racks 106 a-x, 108 a-x and the sets of power buses 110a-x, 112 a-x, 114 a-x, 116 a-x shown in FIG. 1 in order to produce adistributed, redundant configuration. In other words, a distributed,redundant configuration can be achieved by connecting each server rackin the various sets of server racks 106 a-x, 108 a-x to two differentpower buses similarly to the way that the server racks 106 a(1)-106a(20) shown in FIG. 1A are connected to two different power buses (i.e.,the power buses 110 a, 112 a).

The server racks 106 a(1)-106 a(20) shown in FIG. 1A are an example ofthe set of server racks 106 a in the data center 100 shown in FIG. 1.Although the server racks 106 a(1)-106 a(20) shown in FIG. 1A areconnected to cell A and cell B, in the depicted embodiment other sets ofserver racks 106 b-x, 108 a-x in the data center 100 would be connectedto different combinations of cells. For example, each of the serverracks in the set of server racks 108 m can be connected to both thepower bus 114 u that is connected to the power supply system 102 c ofcell C and the power bus 116 m that is connected to the power supplysystem 102 d of cell D. As another example, each of the server racks inthe set of server racks 108 o can be connected to both the power bus 112u that is connected to the power supply system 102 b of cell B and thepower bus 116 o that is connected to the power supply system 102 d ofcell D. Other combinations are also possible, as shown in FIG. 1.

As noted previously, one or more of the power supply systems within adata center's electrical infrastructure can become unavailable from timeto time (e.g., due to component failure and/or scheduled maintenance).The distributed, redundant configuration just described makes itpossible for servers to continue to receive power when a power supplysystem becomes unavailable. In other words, because each server rackdraws power from at least two different power supply systems, thedistributed, redundant configuration enables load shifting. Therefore,if a power supply system becomes unavailable, the load that was beingprovided by the now unavailable power supply system can be shifted toone or more other power supply systems, so that the servers that werebeing supplied by the now unavailable power supply system can continueto receive power.

As an example, consider the loss of the power supply system 102 c incell C. When the power supply system 102 c in cell C is operational, itsupplies power to several sets of server racks 106 g, 106 h, 106 i, 106l, 106 n, 106 s, 106 t, 106 u, 108 a, 108 b, 108 c, 108 d, 108 e, 108 f,108 g, 108 h, 108 i, 108 j, 108 k, 108 l, 108 m, 108 p, 108 q, 108 r inthe data center shown in FIG. 1. When the power supply system 102 c incell C becomes unavailable, the load that the power supply system 102 cin cell C was supplying to these sets of server racks 106 g, 106 h, 106i, 106 l, 106 n, 106 s, 106 t, 106 u, 108 a, 108 b, 108 c, 108 d, 108 e,108 f, 108 g, 108 h, 108 i, 108 j, 108 k, 108 l, 108 m, 108 p, 108 q,108 r can be transferred from the power supply system 102 c in cell C toother power supply systems 102 a, 102 b, 102 d in other cells.

For example, during normal operation the set of server racks 106 greceives power from cell A (via a power bus 110 g that is connected tothe power supply system 102 a of cell A) and from cell C (via a powerbus 114 a that is connected to the power supply system 102 c of cell C).When the power supply system 102 c in cell C becomes unavailable, theload that the power supply system 102 c in cell C was supplying to theset of server racks 106 g can be shifted to the power supply system 102a of cell A.

As another example, during normal operation the set of server racks 106n receives power from cell C (via a power bus 114 e that is connected tothe power supply system 102 c of cell C) and from cell B (via a powerbus 112 f that is connected to the power supply system 102 b of cell B).When the power supply system 102 c in cell C becomes unavailable, theload that the power supply system 102 c in cell C was supplying to theset of server racks 106 n can be shifted to the power supply system 102b of cell B.

In a similar manner, load shifting can be performed with respect to theother sets of server racks 106 h, 106 i, 106 l, 106 s, 106 t, 106 u, 108a, 108 b, 108 c, 108 d, 108 e, 108 f, 108 g, 108 h, 108 i, 108 j, 108 k,108 l, 108 m, 108 p, 108 q, 108 r that were receiving power from thepower supply system 102 c of cell C before that power supply systembecame unavailable.

Load shifting in the manner just described involves increasing theamount of power that is supplied by the power supply systems 102 a, 102b, 102 d in other cells. As discussed above, to reduce the likelihoodthat load shifting would cause any of the components of the power supplysystems 102 a, 102 b, 102 d in the other cells to become overloaded, theelectrical infrastructure can be designed so that the power supplysystems 102 a-d of the various cells operate with a certain amount ofreserve capacity under normal circumstances. When a power supply system(e.g., the power supply system 102 c in cell C) fails and the load thatwas being supplied by the now unavailable power supply system is shiftedto other power supply systems (e.g., the power supply systems 102 a, 102b, 102 d in other cells), these other power supply systems can utilizetheir reserve capacity to supply the additional load.

Consider a numerical example. Suppose that the total capacity of each ofthe power supply systems 102 a-d in each of the cells is 1.2 MW. Becausethere are four cells, this means that the total capacity is 4.8 MW inthis example. Further suppose that the utilization of the data center100 is limited so that the servers' total power consumption does notexceed 3.6 MW. Therefore, the amount of power supplied by the powersupply systems 102 a-d in each of the cells can be limited to 0.9 MWduring normal operation. This leaves a reserve capacity of 0.3 MW foreach of the cells, and a total reserve capacity of 1.2 MW for the entiresystem (which is equal to the total capacity of one of the power supplysystems 102 a-d in one of the cells).

Having this much reserve capacity allows the electrical infrastructureof the data center 100 to maintain smooth operation and prevent outagesfrom occurring when one of the power supply systems becomes unavailable.Continuing with the previous example in which the power supply system102 c in cell C becomes unavailable, consider that the power supplysystem 102 c in cell C was previously supplying 0.9 MW of power beforebecoming unavailable. When the power supply system 102 c in cell Cbecomes unavailable, the other power supply systems 102 a, 102 b, 102 din the other cells can increase the amount of power that they supplyfrom 0.9 MW to 1.2 MW (which their total capacity) in order tocompensate for the loss of power from the power supply system 102 c incell C. Because each of the three other cells increases the amount ofpower that it supplies by 0.3 MW, the total amount of power that issupplied by these other cells increases by 0.9 MW. Therefore, increasingthe amount of power that is supplied by the power supply systems 102 a,102 b, 102 d in these other cells compensates for the loss of power fromthe power supply system 102 c in cell C.

As can be seen from this example, one benefit of the distributed,redundant configuration is that it is possible for servers to continueto receive power when a power supply system becomes unavailable.However, this higher reliability comes at the cost of lower capacity. Inthe example just described, the power supply systems 102 a-d in thevarious cells are each capable of supplying 1.2 MW of power, but theyonly supply 0.9 MW during normal operation in order to maintain somereserve capacity in the event that load shifting becomes necessary.

In scenarios where reliability is more important than capacity, thedistributed, redundant configuration can be beneficial. However, thereare some scenarios in which capacity can be more important thanreliability. Advantageously, the arrangement of power supply systems 102a-d, sets of server racks 106 a-x, 108 a-x, and sets of power buses 110a-x, 112 a-x, 114 a-x, 116 a-x shown in FIG. 1 makes it possible toeasily change from the distributed, redundant configuration to anon-redundant configuration that provides higher capacity. Changing fromthe distributed, redundant configuration to the non-redundantconfiguration simply involves changing the connections between the setsof server racks 106 a-x, 108 a-x and the sets of power buses 110 a-x,112 a-x, 114 a-x, 116 a-x. As noted above, if the sets of server racks106 a-x, 108 a-x and the power buses 110 a-x, 112 a-x, 114 a-x, 116 a-xare connected in accordance with a first connection pattern, thisproduces the distributed, redundant configuration. Alternatively, if thesets of server racks 106 a-x, 108 a-x and the power buses 110 a-x, 112a-x, 114 a-x, 116 a-x are connected in accordance with a secondconnection pattern, this produces the non-redundant configuration. Inthe distributed, redundant configuration, each server rack in thevarious sets of server racks 106 a-x, 108 a-x is connected to twodifferent power buses. For example, as shown in FIG. 1A, each of theserver racks 106 a(1)-106 a(20) is connected to two different powerbuses 110 a, 112 a. In contrast, in the non-redundant configuration,each server rack in the various sets of server racks 106 a-x, 108 a-x isconnected to just one power bus.

FIG. 1B illustrates an example showing how the first set of server racks106 a can be connected to the power buses 110 a, 112 a in accordancewith a second connection pattern in order to produce the non-redundantconfiguration.

In the depicted example, each of the server racks 106 a(1)-106 a(20)includes two distinct connections to the same power bus. For example,referring to the server rack 106 a(1) that is located at the top of thefirst set of server racks 106 a, there are two distinct connections (afirst connection 120 a and a second connection 120 b′) between thatserver rack 106 a(1) and the power bus 112 a that is connected to thepower supply system 102 b of cell B. However, that server rack 106 a(1)is not connected to the power bus 110 a that is connected to the powersupply system 102 a of cell A. Similarly, referring to the next serverrack 106 a(2), there are two distinct connections (a first connection118 a′ and a second connection 118 b) between that server rack 106 a(2)and the power bus 110 a that is connected to the power supply system 102a of cell A. However, that server rack 106 a(2) is not connected to thepower bus 112 a that is connected to the power supply system 102 b ofcell B.

In a similar manner, each of the server racks 106 a(3), 106 a(5), 106a(7), 106 a(9), 106 a(11), 106 a(13), 106 a(15), 106 a(17), 106 a(19)includes two distinct connections to the power bus 112 a that isconnected to the power supply system 102 b of cell B. In particular, theserver rack 106 a(3) includes connections 120 c, 120 d′ to the power bus112 a, the server rack 106 a(5) includes connections 120 e, 120 f to thepower bus 112 a, the server rack 106 a(7) includes connections 120 g,120 h′ to the power bus 112 a, the server rack 106 a(9) includesconnections 120 i, 120 j′ to the power bus 112 a, the server rack 106a(11) includes connections 120 k, 1201′ to the power bus 112 a, theserver rack 106 a(13) includes connections 120 m, 120 n′ to the powerbus 112 a, the server rack 106 a(15) includes connections 120 o, 120 p′to the power bus 112 a, the server rack 106 a(17) includes connections120 q, 120 r′ to the power bus 112 a, and the server rack 106 a(19)includes connections 120 s, 120 t′ to the power bus 112 a. However, noneof the server racks 106 a(3), 106 a(5), 106 a(7), 106 a(9), 106 a(11),106 a(13), 106 a(15), 106 a(17), 106 a(19) are connected to the powerbus 110 a that is connected to the power supply system 102 a of cell A.

Conversely, each of the server racks 106 a(4), 106 a(6), 106 a(8), 106a(10), 106 a(12), 106 a(14), 106 a(16), 106 a(18), 106 a(20) includestwo distinct connections to the power bus 110 a that is connected to thepower supply system 102 a of cell A. In particular, the server rack 106a(4) includes connections 118 c′, 118 d to the power bus 110 a, theserver rack 106 a(6) includes connections 118 e′, 118 f to the power bus110 a, the server rack 106 a(8) includes connections 118 g′, 118 h tothe power bus 110 a, the server rack 106 a(10) includes connections 118i′, 118 j to the power bus 110 a, the server rack 106 a(12) includesconnections 118 k′, 1181 to the power bus 110 a, the server rack 106a(14) includes connections 118 m′, 118 n to the power bus 110 a, theserver rack 106 a(16) includes connections 118 o′, 118 p to the powerbus 110 a, the server rack 106 a(18) includes connections 118 q′, 118 rto the power bus 110 a, and the server rack 106 a(20) includesconnections 118 s′, 118 t to the power bus 110 a. However, none of theserver racks 106 a(4), 106 a(6), 106 a(8), 106 a(10), 106 a(12), 106a(14), 106 a(16), 106 a(18), 106 a(20) are connected to the power bus112 a that is connected to the power supply system 102 b of cell B.

Adjacent server racks in the row of server racks 106 a(1)-106 a(20) areconnected to different power buses. For example, the server racks 106a(1), 106 a(2) are adjacent to one another, and they are connected todifferent power buses (the server rack 106 a(1) is connected to thepower bus 112 a, and the server rack 106 a(2) is connected to the powerbus 110 a). As another example, the server racks 106 a(2), 106 a(3) areadjacent to one another, and they are connected to different power buses(the server rack 106 a(2) is connected to the power bus 110 a, and theserver rack 106 a(3) is connected to the power bus 112 a). In otherwords, every other server rack in the row of server racks 106 a(1)-106a(20) is connected to a different power bus, alternating between thepower bus 110 a and the power bus 112 a.

The connection pattern shown in FIG. 1B between the first set of serverracks 106 a and the power buses 110 a, 112 a is representative of theconnection pattern that can be used to connect the sets of server racks106 a-x, 108 a-x and the sets of power buses 110 a-x, 112 a-x, 114 a-x,116 a-x shown in FIG. 1 in order to produce the non-redundantconfiguration. In other words, the non-redundant configuration can beachieved by connecting each server rack in the various sets of serverracks 106 a-x, 108 a-x to a single power bus similarly to the way thatthe server racks 106 a(1)-106 a(20) shown in FIG. 1B are connected to asingle power bus (either the power bus 110 a that is connected to thepower supply system 102 a of cell A, or the power bus 112 b that isconnected to the power supply system 102 b of cell B).

Of course, the non-redundant configuration shown in FIG. 1B is just oneexample and should not be interpreted as limiting the scope of thepresent disclosure. There are other ways that the server racks 106a(1)-106 a(20) can be connected to the power buses 110 a, 112 a in orderto create a non-redundant configuration.

As noted previously, the server racks 106 a(1)-106 a(20) shown in FIG.1A are an example of the set of server racks 106 a in the data center100 shown in FIG. 1. Although each of the server racks 106 a(1)-106a(20) shown in FIG. 1B are connected to cell A or cell B, in thedepicted embodiment other sets of server racks 106 b-x, 108 a-x in thedata center 100 would be connected to different combinations of cells.For example, each of the server racks in the set of server racks 108 mcan be connected to either the power bus 114 u that is connected to thepower supply system 102 c of cell C or the power bus 116 m that isconnected to the power supply system 102 d of cell D. As anotherexample, each of the server racks in the set of server racks 108 o canbe connected to either the power bus 112 u that is connected to thepower supply system 102 b of cell B or the power bus 116 o that isconnected to the power supply system 102 d of cell D. Other combinationsare also possible, as shown in FIG. 1.

In the non-redundant configuration, it is not necessary to maintain anyreserve capacity. Therefore, the power supply systems 102 a-d in all ofthe cells can operate at full capacity under normal conditions. In theexample described previously, it was assumed that the total capacity ofeach of the power supply systems 102 a-d in each of the cells is 1.2 MW.Therefore, in the non-redundant configuration, the power supply systems102 a-d in each of the cells can supply 1.2 MW of power during normaloperation (instead of supplying 0.9 MW of power and leaving 0.3 MW asreserve capacity, as is the case with the distributed, redundantconfiguration).

The non-redundant configuration does not facilitate load shifting.Therefore, when one of the power supply systems becomes unavailable,outages can occur. However, the arrangement of power supply systems 102a-d, sets of server racks 106 a-x, 108 a-x, and sets of power buses 110a-x, 112 a-x, 114 a-x, 116 a-x shown in FIG. 1 limits the extent of theoutages so that only some of the servers are affected. In particular,the extent of the outages is limited to those servers that werepreviously receiving power from the now unavailable power supply system.

In the example described previously, the loss of the power supply system102 c in cell C was considered. As noted previously, when the powersupply system 102 c in cell C is operational, it supplies power toseveral sets of server racks 106 g, 106 h, 106 i, 106 l, 106 n, 106 s,106 t, 106 u, 108 a, 108 b, 108 c, 108 d, 108 e, 108 f, 108 g, 108 h,108 i, 108 j, 108 k, 108 l, 108 m, 108 p, 108 q, 108 r in the datacenter 100 shown in FIG. 1. If the power supply system 102 c in cell Cbecomes unavailable, then these sets of server racks 106 g, 106 h, 106i, 106 l, 106 n, 106 s, 106 t, 106 u, 108 a, 108 b, 108 c, 108 d, 108 e,108 f, 108 g, 108 h, 108 i, 108 j, 108 k, 108 l, 108 m, 108 p, 108 q,108 r can lose power and experience an outage. However, none of theother sets of server racks in the data center 100 shown in FIG. 1 wouldbe affected by the loss of the power supply system 102 c in cell C.

FIG. 2 illustrates an example of a method 200 that can be performed inorder to facilitate different configurations (a distributed, redundantconfiguration and a non-redundant configuration) for the electricalinfrastructure that is utilized in a section of a data center. Themethod 200 will be described in relation to the data center 100 shown inFIG. 1 and the examples of connection patterns shown in FIGS. 1A and 1B.

The method 200 includes providing 202 a plurality of power supplysystems 102 a-d, a plurality of sets of server racks 106 a-x, 108 a-x,and a plurality of power buses 110 a-x, 112 a-x, 114 a-x, 116 a-x. Asdiscussed above, the power supply systems 102 a-d are configured tosupply electrical power to a section of a data center 100. The powerbuses 110 a-x are configured to conduct the electrical power from thepower supply systems 102 a-d to the server racks 106 a-x, 108 a-x.

The sets of server racks 106 a-x, 108 a-x and the power buses 110 a-x,112 a-x, 114 a-x, 116 a-x can be arranged 204 so that each set of serverracks is accessible to two power buses that are connected to differentpower supply systems. For example, in the data center 100 shown in FIG.1, the first set of server racks 106 a is accessible to a power bus 110a that is connected to the power supply system 102 a of cell A and apower bus 112 a that is connected to the power supply system 102 b ofcell B.

The method 200 can include determining 206 the relative importance ofserver reliability compared to power capacity. If server reliability ismore important than power capacity, then the method 200 can includeconnecting 208 the sets of server racks 106 a-x, 108 a-x and the powerbuses 110 a-x, 112 a-x, 114 a-x, 116 a-x in a distributed, redundantconfiguration. More specifically, the sets of server racks 106 a-x, 108a-x and the power buses 110 a-x, 112 a-x, 114 a-x, 116 a-x can beconnected 208 in accordance with a first connection pattern in whicheach server rack is connected to two different power buses that areconnected to two different power supply systems. An example of this typeof connection pattern is shown in FIG. 1A, in which each of the serverracks 106 a(1)-106 a(20) is connected to both the power bus 110 a thatis connected to the power supply system 102 a of cell A and the powerbus 112 a that is connected to the power supply system 102 b of cell B.

On the other hand, if it is determined 206 that power capacity is moreimportant than server reliability, then the method 200 can includeconnecting 210 the sets of server racks 106 a-x, 108 a-x and the powerbuses 110 a-x, 112 a-x, 114 a-x, 116 a-x in a non-redundantconfiguration. More specifically, the sets of server racks 106 a-x, 108a-x and the power buses 110 a-x, 112 a-x, 114 a-x, 116 a-x can beconnected 210 in accordance with a second connection pattern in whicheach server rack is connected to only one power bus. An example of thistype of connection pattern is shown in FIG. 1B, in which some of theserver racks 106 a(4), 106 a(6), 106 a(8), 106 a(10), 106 a(12), 106a(14), 106 a(16), 106 a(18), 106 a(20) are connected to the power supplysystem 102 a of cell A via the power bus 110 a, while other server racks106 a(3), 106 a(5), 106 a(7), 106 a(9), 106 a(11), 106 a(13), 106 a(15),106 a(17), 106 a(19) are connected to the power supply system 102 b ofcell B via the power bus 112 a.

Advantageously, the arrangement of power supply systems 102 a-d, sets ofserver racks 106 a-x, 108 a-x, and sets of power buses 110 a-x, 112 a-x,114 a-x, 116 a-x disclosed herein makes it possible to switch betweenthe distributed, redundant configuration and the non-redundantconfiguration without any internal changes to the data center 100 otherthan changing the connections between the sets of server racks 106 a-x,108 a-x and the power buses 110 a-x, 112 a-x, 114 a-x, 116 a-x.Therefore, the method 200 can include determining 212 whether aconfiguration change would be desirable. If so, then the method 200 caninclude transitioning 214 between the distributed, redundantconfiguration and the non-redundant configuration. For example, if atone point in time reliability is more important than capacity, a sectionof a data center 100 can initially be set up in the distributed,redundant configuration. If at a later point in time capacity becomesmore important than reliability, the section of the data center 100 canbe transitioned 214 from the distributed, redundant configuration to thenon-redundant configuration simply by changing the connections betweenthe sets of server racks 106 a-x, 108 a-x and the power buses 110 a-x,112 a-x, 114 a-x, 116 a-x. Alternatively, a transition could be madefrom the non-redundant configuration to the distributed, redundantconfiguration if reliability becomes more important than capacity.

FIG. 3 illustrates examples of components that enable the power supplysystems 302 a-d of various cells to be connected so as to facilitate adistributed, redundant configuration. As before, it will be assumed thatthere are four cells that are configured to supply power to servers in aparticular section of the data center. These cells will once again bereferred to as cell A, cell B, cell C, and cell D. Cell A includes apower supply system 302 a, cell B includes a power supply system 302 b,cell C includes a power supply system 302 c, and cell D includes a powersupply system 302 d. The system 300 includes a plurality of switchboards346 a-d.

A server 356 a is shown in FIG. 3. In addition to receiving electricalpower from the power supply system 302 a of cell A, the server 356 aalso receives electrical power from the power supply system 302 b ofcell B. In particular, the server 356 a receives electrical power fromthe power supply system 302 a of cell A via the first switchboard 356 a,and the server 356 b receives electrical power from the power supplysystem 302 b of cell B via the second switchboard 356 b. Although forthe sake of simplicity only one server 356 a is shown in FIG. 3, thepower supply systems 302 a-d can, of course, supply power to a pluralityof servers. Each server can be configured to draw power from at leasttwo different power supply systems when the overall system is connectedin accordance with a distributed, redundant configuration.

In addition to being connected to the power supply system 302 a of cellA, the first switchboard 346 a also includes connections to the powersupply system 302 b of cell B, the power supply system 302 c of cell C,and the power supply system 302 c of cell D. If the power supply system302 a of cell A should fail or otherwise become unavailable, theswitchboard 346 a can transfer the server 356 a (and other servers beingsupplied by cell A) to the power supply systems 302 b, 302 c, 302 d ofthe B, C, and D cells.

Like the first switchboard 346 a, the other switchboards 346 b-d arealso connected to the power supply systems 302 a-d of each of the cells.If the power supply system 302 b of cell B becomes unavailable, thesecond switchboard 346 b can transfer the servers being supplied by thepower supply system 302 b of cell B to the power supply systems 302 a,302 c, 302 d of the A, C, and D cells. If the power supply system 302 cof cell C becomes unavailable, the third switchboard 346 c can transferthe servers being supplied by the power supply system 302 c of cell C tothe power supply systems 302 a, 302 b, 302 d of the A, B, and D cells.If the power supply system 302 d of cell D becomes unavailable, thefourth switchboard 346 d can transfer the servers being supplied by thepower supply system 302 d of cell D to the power supply systems 302 a,302 b, 302 c of the A, B, and C cells.

In some embodiments, different sections of a data center can utilizedifferent configurations. For example, one section of a data center canbe configured in accordance with a distributed, redundant configuration,while another section of the data center can be configured in accordancewith a non-redundant configuration.

FIG. 4 illustrates an example of a data center 400 in which differentsections utilize different configurations. In the depicted example, thedata center 400 includes a first section 458 a and a second section 458b. The first section 458 a of the data center 400 includes four powersupply systems 402 a(1), 402 a(2), 402 a(3), 402 a(4). Similarly, thesecond section 458 b of the data center 400 includes four power supplysystems 402 b(1), 402 b(2), 402 b(3), 402 b(4). Both sections 458 a, 458b of the data center 400 also include a plurality of sets of serverracks 406 a, 406 b, and a plurality of power buses 410 a, 410 b that areconfigured to conduct electrical power from the power supply systems 402a, 402 b to the sets of server racks 406 a, 406 b.

The different sections 458 a-b of the data center 400 can utilizedifferent configurations. For example, the first section 458 a of thedata center 400 can be configured in accordance with a distributed,redundant configuration in which each server rack in the various sets ofserver racks 406 a is connected to two different power buses 410 a,similarly to the way that the server racks 106 a(1)-106 a(20) shown inFIG. 1A are connected to two different power buses 110 a, 112 a. On theother hand, the second section 458 b of the data center 400 can beconfigured in accordance with a non-redundant configuration in whicheach server rack in the various sets of server racks 406 b is connectedto only one power bus 410 b, similarly to the way that the server racks106 a(1)-106 a(20) shown in FIG. 1B are connected to a single power bus(either the power bus 110 a or the power bus 112 b). These differentconfigurations can be achieved in the different sections 458 a, 458 b ofthe data center 400 by the way that the connections between the sets ofserver racks 406 a, 406 b and the power buses 110 a, 112 b areconfigured, as described above.

The steps, operations, and/or actions of the methods described hereinmay be interchanged with one another without departing from the scope ofthe claims. In other words, unless a specific order of steps,operations, and/or actions is required for proper functioning of themethod that is being described, the order and/or use of specific steps,operations, and/or actions may be modified without departing from thescope of the claims.

The term “determining” (and grammatical variants thereof) can encompassa wide variety of actions. For example, “determining” can includecalculating, computing, processing, deriving, investigating, looking up(e.g., looking up in a table, a database or another data structure),ascertaining and the like. Also, “determining” can include receiving(e.g., receiving information), accessing (e.g., accessing data in amemory) and the like. Also, “determining” can include resolving,selecting, choosing, establishing and the like.

The terms “comprising,” “including,” and “having” are intended to beinclusive and mean that there can be additional elements other than thelisted elements. Additionally, it should be understood that referencesto “one embodiment” or “an embodiment” of the present disclosure are notintended to be interpreted as excluding the existence of additionalembodiments that also incorporate the recited features. For example, anyelement or feature described in relation to an embodiment herein may becombinable with any element or feature of any other embodiment describedherein, where compatible.

The present disclosure may be embodied in other specific forms withoutdeparting from its spirit or characteristics. The described embodimentsare to be considered as illustrative and not restrictive. The scope ofthe disclosure is, therefore, indicated by the appended claims ratherthan by the foregoing description. Changes that come within the meaningand range of equivalency of the claims are to be embraced within theirscope.

What is claimed is:
 1. A system, comprising: a plurality of power supplysystems that are configured to supply electrical power to at least asection of a data center; a first plurality of sets of server racks; afirst plurality of power buses that are configured to conduct theelectrical power from at least some of the plurality of power supplysystems to the first plurality of sets of server racks, wherein thefirst plurality of sets of server racks and the first plurality of powerbuses are connected in accordance with a first connection pattern tocreate a distributed, redundant configuration; a second plurality ofsets of server racks; and a second plurality of power buses that areconfigured to conduct the electrical power from at least some of theplurality of power supply systems to the second plurality of sets ofserver racks, wherein the second plurality of sets of server racks andthe second plurality of power buses are connected in accordance with asecond connection pattern to create a non-redundant configuration. 2.The system of claim 1, wherein the distributed, redundant configurationprovides higher server reliability than the non-redundant configuration.3. The system of claim 1, wherein the non-redundant configurationprovides higher power capacity than the distributed, redundantconfiguration.
 4. The system of claim 1, wherein each server rack in thefirst plurality of sets of server racks is connected to two differentpower supply systems.
 5. The system of claim 1, wherein each server rackin the second plurality of sets of server racks is connected to only onepower supply system.
 6. The system of claim 1, wherein the plurality ofpower supply systems comprise: a first plurality of power supply systemsthat maintain reserve capacity during normal operation; and a secondplurality of power supply systems that operate at full capacity withoutmaintaining any reserve capacity.